Songle Sync: A Large-Scale Web-based Platform for Controlling Various Devices in Synchronization with Music

インターネットに接続された100個のデバイスを音楽再生と同期するように制御できる、ウェブベースプラットフィーム Songle Syncを提案。

3つの問題に取り組む

- Scalability

- Variety

- Programming experience

system provide end-user with more sense of music and unity with other people

システムはvariety of networking environmentsの下でheterogeneous hardware setup をサポート。そのためalways-on architectureは採用できない。

Evaluation Strategies for HCI Toolkit Researchを参考に評価実験設計

- パフォーマンス調査

- 実環境テスト(１００個デバイス繋いでデモとマジカルミライでの実施)

- ハッカソン usability調査

様々な利用シーンを述べて、それらをIoMTと名付けてまとめた。

実環境やハッカソンから得られた知見をまとめてる。

--------------------------------------------------------------------------------------------------------

Abstract

This paper presents Songle Sync, a web-based platform on which hundreds of Internet-connected devices - including smartphones, computers, and other physical computing devices - can be controlled to synchronize with music playback. It uses music-understanding technologies to dynamically synthesize music-driven multimedia performances from a musical piece of choice. To simultaneously control hundreds of devices, a conventional architecture keeps always-on connections between them. However, it does not scale and suffers from latency and jitter issues when there are various devices with potentially unstable networks. We address this with a novel autonomous control architecture in which each device is notified of forthcoming musical events (e.g., beats and chorus sections) to automatically drive various changes in multimedia performances. Moreover, we provide a development kit of an event-driven multimedia framework for JavaScript, example programs, and an interactive tutorial. To evaluate the platform, we compared latencies, jitters, and amounts of network traffic between ours and the conventional architecture. To examine use cases in the wild, we deployed the platform to drive over a hundred of a variety of devices. We also developed a web browser-based application for a multimedia performance with music playback. It provided audiences of hundreds with a bring-your-own-device experience of synchronized animations on smartphones. In addition, the development kit was used in a two-day hackathon. We report lessons learned from these studies and discuss the future of the Internet of Musical Things.

@inproceedings{Kato:2018:SSL:3240508.3240619,

author = {Kato, Jun and Ogata, Masa and Inoue, Takahiro and Goto, Masataka},

title = {Songle Sync: A Large-Scale Web-based Platform for Controlling Various Devices in Synchronization with Music},

booktitle = {Proceedings of the 26th ACM International Conference on Multimedia},

isbn = {978-1-4503-5665-7},

location = {Seoul, Republic of Korea},

pages = {1697--1705},

numpages = {9},

doi = {10.1145/3240508.3240619},

acmid = {3240619},

publisher = {ACM},

address = {New York, NY, USA},

keywords = {application programming interface, internet of musical things, multimedia control, music synchronization},

}